Hybrid unsupervised and supervised image segmentation model

ABSTRACT

Systems and techniques that facilitate hybrid unsupervised and supervised image segmentation are provided. In various embodiments, a system can access a computed tomography (CT) image depicting an anatomical structure. In various aspects, the system can generate, via an unsupervised modeling technique, at least one class probability mask of the anatomical structure based on the CT image. In various instances, the system can generate, via a deep-learning model, an image segmentation based on the CT image and based on the at least one class probability mask.

TECHNICAL FIELD

The subject disclosure relates generally to image segmentation, and morespecifically to a hybrid un-supervised and supervised image segmentationmodel.

BACKGROUND

Measurement and/or estimation of hemodynamics in vascular structures isuseful for assessing cardiovascular disease. In existing clinicalpractice, hemodynamics within a vascular structure can be physicallymeasured via a catheter that is fitted with appropriate transducers.Unfortunately, such physical measurement is highly invasive,time-consuming, expensive, and localized. To address such issues,existing clinical practice has begun to incorporate non-invasiveestimation of hemodynamics based on computed tomography (CT) scans thatdepict vascular structures of interest. Properly estimating thehemodynamics of a vascular structure that is depicted in a CT scandepends upon the CT scan being accurately segmented. Existingdeep-learning models can facilitate segmentation of CT scans. However,such existing deep-learning models require huge volumes of training dataand often fail to properly generalize.

Accordingly, systems and/or techniques that can address one or more ofthese technical problems can be desirable.

SUMMARY

The following presents a summary to provide a basic understanding of oneor more embodiments of the invention. This summary is not intended toidentify key or critical elements, or delineate any scope of theparticular embodiments or any scope of the claims. Its sole purpose isto present concepts in a simplified form as a prelude to the moredetailed description that is presented later. In one or more embodimentsdescribed herein, devices, systems, computer-implemented methods,apparatus and/or computer program products that facilitate hybridunsupervised and supervised image segmentation are described.

According to one or more embodiments, a system is provided. The systemcan comprise a computer-readable memory that can storecomputer-executable components. The system can further comprise aprocessor that can be operably coupled to the computer-readable memoryand that can execute the computer-executable components stored in thecomputer-readable memory. In various embodiments, thecomputer-executable components can comprise a receiver component. Invarious aspects, the receiver component can access a computed tomography(CT) image depicting an anatomical structure. In various instances, thecomputer-executable components can further comprise a probabilitycomponent. In various cases, the probability component can generate, viaan unsupervised modeling technique, at least one class probability maskof the anatomical structure based on the CT image. In various aspects,the computer-executable components can further comprise an executioncomponent. In various instances, the execution component can generate,via a deep-learning model, an image segmentation based on the CT imageand based on the at least one class probability mask.

According to one or more embodiments, the above-described system can beimplemented as a computer-implemented method and/or a computer programproduct.

DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of an example, non-limiting systemthat facilitates hybrid unsupervised and supervised image segmentationin accordance with one or more embodiments described herein.

FIG. 2 illustrates a block diagram of an example, non-limiting systemincluding Gaussian mixture modeling and class probability masks thatfacilitates hybrid unsupervised and supervised image segmentation inaccordance with one or more embodiments described herein.

FIG. 3 illustrates an example, non-limiting Hounsfield unit probabilitydistribution exhibited by an example, non-limiting computed tomographyimage or volume in accordance with one or more embodiments describedherein.

FIG. 4 illustrates how an example, non-limiting Hounsfield unitprobability distribution can be the sum of multiple Gaussiandistributions in accordance with one or more embodiments describedherein.

FIG. 5 illustrates an example, non-limiting computed tomography image inaccordance with one or more embodiments described herein.

FIG. 6 illustrates example, non-limiting class probability masksgenerated via Gaussian mixture modeling in accordance with one or moreembodiments described herein.

FIG. 7 illustrates a block diagram of an example, non-limiting systemincluding a deep-learning model and a segmentation that facilitateshybrid unsupervised and supervised image segmentation in accordance withone or more embodiments described herein.

FIG. 8 illustrates a block diagram of an example, non-limiting systemincluding a training component that facilitates hybrid unsupervised andsupervised image segmentation in accordance with one or more embodimentsdescribed herein.

FIG. 9 illustrates a flow diagram of an example, non-limitingcomputer-implemented method that facilitates hybrid unsupervised andsupervised image segmentation in accordance with one or more embodimentsdescribed herein.

FIG. 10 illustrates a flow diagram of an example, non-limitingcomputer-implemented method that facilitates hybrid unsupervised andsupervised image segmentation in accordance with one or more embodimentsdescribed herein.

FIG. 11 illustrates a block diagram of an example, non-limitingoperating environment in which one or more embodiments described hereincan be facilitated.

FIG. 12 illustrates an example networking environment operable toexecute various implementations described herein.

DETAILED DESCRIPTION

The following detailed description is merely illustrative and is notintended to limit embodiments and/or application or uses of embodiments.Furthermore, there is no intention to be bound by any expressed orimplied information presented in the preceding Background or Summarysections, or in the Detailed Description section.

One or more embodiments are now described with reference to thedrawings, wherein like referenced numerals are used to refer to likeelements throughout. In the following description, for purposes ofexplanation, numerous specific details are set forth in order to providea more thorough understanding of the one or more embodiments. It isevident, however, in various cases, that the one or more embodiments canbe practiced without these specific details.

Cardiovascular disease claims more lives each year than all forms ofcancer and chronic lower respiratory diseases combined. Indeed,cardiovascular disease accounted for over 859,000 deaths in the UnitedStates and 17.8 million deaths worldwide in 2017 alone. Accordingly,techniques for identifying, diagnosing, and/or otherwise assessingcardiovascular disease can be desirable.

Cardiovascular disease in a patient can be assessed by measuring and/orestimating the hemodynamics (e.g., blood pressure, blood flow,fractional flow reserve (FFR)) occurring within the patient's vascularstructures (e.g., blood vessels, arteries, lumina).

In existing clinical practice, hemodynamics within a vascular structurecan be physically measured via a catheter that is fitted withtransducers (e.g., pressure and/or flow transducers). Unfortunately,such physical measurement is highly invasive, time-consuming, andexpensive. Moreover, such physical measurement is highly localized. Thatis, the measured hemodynamics apply only to the vascular structure inwhich the catheter is inserted and do not necessarily apply to adjoiningvascular structures. Accordingly, to physically measure the hemodynamicsin such adjoining vascular structures, catheters would have to beinserted into all of such adjoining vascular structures, which is evenmore invasive, time-consuming, expensive, and/or risky to the patient.

To address such issues, existing clinical practice has begun toincorporate non-invasive estimation of hemodynamics based on computedtomography (CT) images that depict vascular structures of interest.Specifically, a CT image of a vascular structure can be generated bymedical imaging equipment (e.g., a CT imaging system), and the CT imagecan be enhanced and/or analyzed via machine learning models (and/oranalytic models) that are configured to estimate, infer, and/orotherwise predict the hemodynamics within the depicted vascularstructure. However, proper estimation, inference, and/or prediction ofthe hemodynamics of the vascular structure that is depicted in the CTimage depends upon the CT image being accurately segmented. In otherwords, if the machine learning models are not able to accurately segmentthe CT image, and/or if an accurate segmentation of the CT image is nototherwise available to the machine learning models (and/or analyticmodels), the machine learning models (and/or analytic models) cannotaccurately estimate the hemodynamics of the vascular structure depictedin the CT image.

Image segmentation of CT images can be performed by deep-learning models(e.g., artificial neural networks). Specifically, a CT image can, invarious cases, be considered as a matrix of pixel-wise Hounsfield unit(HU) values, and a deep-learning model can be trained (e.g., viabackpropagation) to receive the CT image as input and to produce animage segmentation of the CT image as output. The image segmentation canbe a pixel-wise mask of the CT image that indicates to which of two ormore classes each pixel of the CT image belongs. In various cases, thedeep-learning model can be considered as a supervised segmentationmodel. That is, the deep-learning model can be trained in a supervisedfashion on training data to perform image segmentation (e.g., thedeep-learning model can be fed manually annotated CT images, and theparameters of the deep-learning model can be iteratively updated viabackpropagation).

Unfortunately, existing deep-learning models require huge volumes oftraining data to achieve acceptable segmentation accuracy. Such hugevolumes of training data can be highly expensive and time-consuming toprocure, and/or can be even more time-consuming to annotate (e.g.,indeed, existing clinical practice mainly utilizes manual annotation,which is very time intensive). Moreover, even with such voluminoustraining data, existing deep-learning models often fail to properlygeneralize to multi-site settings. For instance, different medical sites(e.g., different hospitals, different hospital departments) can utilizedifferent types and/or amounts of contrast agents, different clinicalprotocols, different imaging equipment, and/or different reconstructionapproaches to create CT images. Such differences can cause different CTimages generated by those different medical sites to exhibit differentHU intensities, which can adversely affect performance of existingdeep-learning models. In other words, an existing deep-learning modelthat has been extensively trained on CT images provided by one medicalsite often can exhibit significantly degraded accuracy when segmentingCT images provided by a different medical site. Because of suchshortcomings, existing clinical practice usually involves extensivemanual editing and/or refinement of image segmentations provided by suchexisting deep-learning models. Such manual post-processing can consumeeven more time and resources (e.g., about four to five hours to manuallyedit/refine CT images). Thus, systems and/or techniques that can addressone or more of these technical problems can be desirable.

Various embodiments of the subject innovation can address one or morethese technical problems. One or more embodiments described hereininclude systems, computer-implemented methods, apparatus, and/orcomputer program products that can facilitate hybrid unsupervised andsupervised image segmentation. In various aspects, the inventors of thesubject innovation recognized that existing deep-learning models thatare trained to perform image segmentation on CT images are structured assingle-channel models. That is, such existing deep-learning models areconfigured to receive as input only the CT image that is desired to besegmented. The inventors realized that such single-channel models arenot constrained by any a priori knowledge about the tissue types and/orthe context of the anatomical structure depicted in the CT image.Accordingly, the inventors devised various embodiments of the subjectinnovation to generate such a priori knowledge via unsupervised modelingtechniques and to provide such a priori knowledge as additional input toa multi-channel deep-learning model that is configured to produce imagesegmentations.

Implementation of such a multi-channel deep-learning model as describedherein can cause various embodiments of the subject innovation togenerate more accurate image segmentations as compared to existingsingle-channel deep-learning models. Furthermore, implementation of amulti-channel deep-learning model as described herein can cause variousembodiments of the subject innovation to require significantly lesstraining data to achieve acceptable segmentation accuracy as compared toexisting single-channel deep-learning models.

In various instances, embodiments of the subject innovation can beconsidered as a computerized tool (e.g., a combination ofcomputer-executable hardware and/or computer-executable software) thatcan electronically receive a CT image that is desired to be segmented.In various aspects, as described herein, the computerized tool canelectronically apply an unsupervised modeling technique to the CT image,thereby yielding one or more class probability masks. In various cases,the computerized tool can electronically feed both the CT image and theone or more class probability masks as input to a deep-learning model,where the deep-learning model is configured to produce an imagesegmentation based on the CT image and based on the one or more classprobability masks. In various instances, if the CT image is an annotatedtraining image, the computerized tool can electronically update, viabackpropagation, internal parameters (e.g., weights, biases) of thedeep-learning model based on a ground truth annotation associated withthe CT image. That is, the deep-learning model can be trained in asupervised fashion. Because the deep-learning model can be trained in asupervised fashion, and because the deep-learning model can beconfigured to receive as input both the CT image and the one or moreclass probability masks that are generated by the unsupervised modelingtechnique, the deep-learning model can be considered as a hybridunsupervised and supervised multi-channel segmentation model. This is instark contrast to a purely supervised single-channel segmentation model,which would be configured to receive as input only the CT image and notthe one or more class probability masks. Although a CT image isdescribed above and in the description that follows, it should beappreciated that the herein teachings can be applied to a collection ofCT images constituting an imaging volume.

As described herein, a hybrid unsupervised and supervised multi-channelsegmentation model can exhibit improved segmentation performance (e.g.,improved accuracy, reduced training time) as opposed to a purelysupervised single-channel segmentation model. Such improved performancecan be due to the one or more class probability masks generated by thecomputerized tool. Indeed, in various cases, the one or more classprobability masks that are generated by the computerized tool via theunsupervised modeling technique can be considered as revealingadditional patterns, trends, distributions, and/or relations that arehidden within the HU values of the CT image. In other words, suchadditional patterns, trends, distributions, and/or relations cannot beeasily discernable by viewing the CT image alone. So, a purelysupervised single-channel segmentation model that is configured toreceive as input only the CT image and not the one or more classprobability masks can have no access to such additional patterns,trends, distributions, and/or relations. On the other hand, a hybridunsupervised and supervised multi-channel segmentation model that isconfigured to receive as input both the CT image and the one or moreclass probability masks can have access to such additional patterns,trends, distributions, and/or relations, which can help to improve theaccuracy/precision of the image segmentation generated by the hybridunsupervised and supervised multi-channel segmentation model. Moreover,because such a hybrid unsupervised and supervised multi-channelsegmentation model has access to such additional patterns, trends,distributions, and/or relations, the hybrid unsupervised and supervisedmulti-channel segmentation model can learn more quickly and can thus betrained in significantly less time and/or with significantly lesstraining data than a purely supervised single-channel segmentationmodel.

In various embodiments, the computerized tool described herein cancomprise a receiver component, a probability component, an executioncomponent, and/or a training component.

In various embodiments, the receiver component of the computerized toolcan electronically receive and/or otherwise electronically access a CTimage. In various cases, the receiver component can electronicallyretrieve the CT image from any suitable centralized and/or decentralizeddata structure (e.g., graph data structure, relational data structure,hybrid data structure), whether remote from and/or local to the receivercomponent. In various other cases, the receiver component canelectronically receive the CT image from any suitable medical imagingequipment (e.g., CT imaging system). In various cases, the CT image candepict a vascular structure of a patient, such as a blood vessel, anartery, and/or a lumen. In various other cases, however, the CT imagecan depict any other suitable anatomical structure (e.g., brain, lung,heart, kidney, intestine, eye, bladder) and/or any suitable portion ofan anatomical structure. In various aspects, the CT image can begenerated by any suitable CT imaging equipment (e.g., any suitablecontrast agent detected by any suitable CT imaging system operatingunder any suitable clinical protocol). Although the herein disclosuremainly describes CT images and CT imaging equipment, it is to beunderstood that any other types of anatomical images and/or imagingmodalities can be implemented (e.g., CT, contrast-enhanced CT, magneticresonance imaging (MRI), ultrasound). In various instances, the CT imagecan be organized as a two-dimensional matrix of pixels with each pixelhaving an associated HU value and/or as a three-dimensional tensor ofvoxels with each voxel having an associated HU value. For ease ofexplanation, the herein disclosure mainly treats the CT image as atwo-dimensional matrix of pixels. However, those having ordinary skillin the art will readily understand that the herein teachings are equallyapplicable to a three-dimensional array of voxels (e.g., athree-dimensional array of voxels can be considered as a sequence oftwo-dimensional pixel matrices).

In various aspects, it can be desired to segment the CT image accordingto n classes, for any suitable positive integer n. That is, it can bedesired to obtain an image segmentation of the CT image, where the imagesegmentation indicates, for each pixel, a particular one of n differentclasses to which the pixel belongs and/or likely belongs. Because the CTimage can depict a vascular structure, the n different classes canpertain to such vascular structure. For example, the n different classescan include a vascular wall class (e.g., for pixels that represent thewall of the vascular structure), a calcification class (e.g., for pixelsthat represent calcified portions of the vascular structure), a bloodclass (e.g., for pixels that represent blood that is flowing through thevascular structure), and/or a background class (e.g., for pixels thatrepresent something other than the wall, calcification, and/or blood ofthe vascular structure). As explained herein, the computerized tool cangenerate such an image segmentation.

In various embodiments, the probability component of the computerizedtool can electronically generate n class probability masks by applyingan unsupervised modeling technique to the CT image. As a non-limitingexample, the unsupervised modeling technique can be Gaussian mixturemodeling. More specifically, the probability component can, in variousaspects, electronically tabulate a probability distribution of HU valuesexhibited by the pixels of the CT image. For example, the probabilitycomponent can define any suitable HU bin size, thereby yielding aparticular number of HU bins (e.g., can be based on the HU rangeexhibited by the CT image). Moreover, the probability component cancount the number of pixels in the CT image that fall within each of theHU bins, thereby yielding a frequency distribution of HU valuesexhibited by the pixels of the CT image, commonly referred to as ahistogram. Furthermore, the probability component can compute theprobability density of each HU bin by dividing the area of the HU bin bythe total area of all of the HU bins, thereby yielding a probabilitydistribution of HU values exhibited by the pixels of the CT image.

In various cases, the probability component can apply Gaussian mixturemodeling to the tabulated probability distribution. As those havingordinary skill in the art will appreciate, Gaussian mixture modeling canbe considered as an iterative algorithm that utilizes any suitableparameter estimation technique (e.g., expectation maximization, Markovchain Monte Carlo, moment matching, spectral method) to iterativelyestimate parameters (e.g., means and/or variances) of multiple Gaussiandistributions that sum to a given probability distribution. Therefore,the probability component can leverage Gaussian mixture modeling so asto model the tabulated probability distribution of the CT image as amixture of n Gaussian distributions. In other words, the probabilitycomponent can identify the HU means and the HU variances of n differentGaussian distributions that collectively sum to the tabulatedprobability distribution. Note that the number of Gaussian distributionsestimated by the probability component can be equal to the number ofsegmentation classes (e.g., n classes, n Gaussian distributions, oneGaussian distribution per class). Note, further, that application ofGaussian mixture modeling can be considered as an unsupervised procedure(e.g., Gaussian mixture modeling is an iterative technique to identifycomponent Gaussians of an overall probability distribution, withoutrequiring any annotated training data).

In various aspects, the computerized tool can generate n classprobability masks based on the n Gaussian distributions. Morespecifically, for each pixel in the CT image, the computerized tool canassign to the pixel n probabilities that respectively correspond to then Gaussian distributions, based on the pixel's HU value. For instance,for every pixel, the probability component can compute a firstlikelihood that the pixel belongs to the first Gaussian distributiongiven the pixel's HU value and given the mean and/or variance of thefirst Gaussian distribution, can compute a second likelihood that thepixel belongs to the second Gaussian distribution given the pixel's HUvalue and given the mean and/or variance of the second Gaussiandistribution, and/or can compute an n-th likelihood that the pixelbelongs to the n-th Gaussian distribution given the pixel's HU value andgiven the mean and/or variance of the n-th Gaussian distribution.

Accordingly, the probability component can output n class probabilitymasks based on these computed probabilities. For example, theprobability component can generate a first class probability maskcomprising the same number of pixels arranged in the same positions asthe CT image, where the value of each pixel in the first classprobability mask is equal to and/or otherwise based on that pixel'slikelihood of belonging to the first Gaussian distribution. So, pixelsthat exhibit high values in the first class probability mask can beconsidered as having high likelihoods of belonging to the first Gaussiandistribution, while pixels that exhibit low values in the first classprobability mask can be considered as having low likelihoods ofbelonging to the first Gaussian distribution.

Similarly, the probability component can generate a second classprobability mask comprising the same number of pixels arranged in thesame positions as the CT image, where the value of each pixel in thesecond class probability mask is equal to and/or otherwise based on thatpixel's likelihood of belonging to the second Gaussian distribution.Thus, pixels that exhibit high values in the second class probabilitymask can be considered as having high likelihoods of belonging to thesecond Gaussian distribution, while pixels that exhibit low values inthe second class probability mask can be considered as having lowlikelihoods of belonging to the second Gaussian distribution.

Likewise, the probability component can generate an n-th classprobability mask comprising the same number of pixels arranged in thesame positions as the CT image, where the value of each pixel in then-th class probability mask is equal to and/or otherwise based on thatpixel's likelihood of belonging to the n-th Gaussian distribution. So,pixels that exhibit high values in the n-th class probability mask canbe considered as having high likelihoods of belonging to the n-thGaussian distribution, while pixels that exhibit low values in the n-thclass probability mask can be considered as having low likelihoods ofbelonging to the n-th Gaussian distribution.

Note that, unlike the CT image, the class probability masks do notexhibit HU values on a pixel-wise basis. Instead, the class probabilitymasks can exhibit probability values on a pixel-wise basis.

Although the above discussion explains how the probability component cangenerate the n class probability masks when the unsupervised modelingtechnique is Gaussian mixture modeling, this is a mere non-limitingexample. Those having ordinary skill in the art will appreciate that theunsupervised modeling technique can be any other suitable procedure forassigning class probabilities to each pixel in the CT image.Non-limiting examples of such other unsupervised modeling techniques caninclude fuzzy C-means clustering, K-means clustering, auto-encodermodeling, generative adversarial networks, and/or any other suitableclustering technique that does not require ground truth labels.

In various embodiments, the execution component of the computerized toolcan electronically feed the CT image and at least one of the n classprobability masks to a deep-learning model, and the deep-learning modelcan electronically produce as output an image segmentation based on theCT image and based on at least one of the n class probability masks. Invarious aspects, the deep-learning model can exhibit any suitable neuralnetwork architecture. For example, the deep-learning model can includeany suitable number of layers, any suitable number of neurons in variouslayers (e.g., different layers can include different numbers ofneurons), any suitable activation functions (e.g., sigmoid, softmax,hyperbolic tangent), and/or any suitable interneuron connectivitypatterns (e.g., forward connections, skip connections, recurrentconnections). As a non-limiting example, the deep-learning model canexhibit a U-Net architecture (e.g., can have an encoder portion thatincludes down-sampling layers, and/or can have a decoder portion thatincludes up-sampling layers). Experimental results confirm that, becausethe deep-learning model can be configured to receive as input the CTimage and at least one of the n class probability masks, rather than theCT image alone, the deep-learning model can, once trained, exhibitbetter segmentation performance (e.g., better accuracy, higher Dicescores) than a deep-learning model that is configured to receive asinput only the CT image. Moreover, experimental results confirm that,because the deep-learning model can be configured to receive as inputthe CT image and at least one of the n class probability masks, ratherthan the CT image alone, the deep-learning model can be trained toaccurately segment CT images in significantly less time and/or withsignificantly less training data than a deep-learning model that isconfigured to receive as input only the CT image.

In various embodiments, if the deep-learning model is not yet trained,the receiver component can electronically access any suitable set oftraining CT images, and the training component of the computerized toolcan electronically train, in a supervised fashion, the deep-learningmodel based on the set of training CT images. More specifically, it canbe the case that each training CT image is annotated with a ground truthsegmentation. In various cases, the internal parameters (e.g., weights,biases) of the deep-learning model can be randomly initialized. Invarious aspects, for each training CT image: the probability componentcan generate n training class probability masks for the training CTimage by leveraging the unsupervised modeling technique; the executioncomponent can feed the both training CT image and at least one of the ntraining class probability masks to the deep-learning model, therebyyielding an image segmentation of the training CT image; and thetraining component can apply backpropagation to update the internalparameters of the deep-learning model, based on differences between theimage segmentation produced by the deep-learning model and the groundtruth annotation of the training CT image. In various cases, thetraining component can implement batch updating of the internalparameters of the deep-learning model, as desired. Because thedeep-learning model can be trained in such a supervised fashion, andbecause the deep-learning model can be configured to receive as inputclass probability masks that are generated via the unsupervised modelingtechnique, the deep-learning model can ultimately be considered as ahybrid unsupervised and supervised image segmentation model.

In summary, the computerized tool described herein can facilitate thecreation and/or deployment of a hybrid unsupervised and supervised imagesegmentation model. Specifically, the computerized tool described hereincan train and/or execute a multi-channel deep-learning model, where themulti-channel deep-learning model can exhibit significantly improvedsegmentation performance as compared to existing single-channel models.In particular, the multi-channel deep-learning model can undergosupervised training, and can be configured to receive as input both a CTimage and one or more class probability masks, where the one or moreclass probability masks can be derived from the CT image via anunsupervised modeling technique (e.g., Gaussian mixture modeling, fuzzyC-means clustering, K-means clustering, auto-encoder modeling,generative adversarial networks). In various cases, the one or moreclass probability masks can be considered as including contextualinformation about the pixels in the CT image (e.g., indicating whichpixels in the CT image are likely to be related to each other), and suchcontextual information can have a regularizing effect on themulti-channel deep-learning model. This regularizing effect is notexperienced by existing single-channel models whose input is notsupplemented with such class probability masks. As a result, themulti-channel deep-learning model can exhibit significantly bettersegmentation accuracy while simultaneously requiring significantly lesstraining data to achieve such segmentation accuracy.

Various embodiments of the subject innovation can be employed to usehardware and/or software to solve problems that are highly technical innature (e.g., to facilitate hybrid unsupervised and supervised imagesegmentation), that are not abstract and that cannot be performed as aset of mental acts by a human. Further, some of the processes performedcan be performed by a specialized computer (e.g., deep-learning models,Gaussian mixture modeling) for carrying out defined tasks related tohybrid unsupervised and supervised image segmentation. For example, suchdefined tasks can include: accessing, by a device operatively coupled toa processor, a computed tomography (CT) image depicting an anatomicalstructure; generating, by the device and via an unsupervised modelingtechnique, at least one class probability mask of the anatomicalstructure based on the CT image; and generating, by the device and via adeep-learning model, an image segmentation based on the CT image andbased on the at least one class probability mask. In various aspects,such defined tasks can further include: accessing, by the device, atraining CT image; generating, by the device and via the unsupervisedmodeling technique, at least one training class probability mask basedon the training CT image; and training, by the device, the deep-learningmodel based on the training CT image and based on the at least onetraining class probability mask.

Such defined tasks are not performed manually by humans. Indeed, neitherthe human mind nor a human with pen and paper can electronically receivea CT scan, electronically implement an unsupervised modeling technique(e.g., Gaussian mixture modeling) on the HU probability distributionexhibited by the CT scan so as to produce one or more class probabilitymasks, electronically feed both the CT scan and the one or more classprobability masks as input to a multi-channel deep-learning segmentationmodel, and/or electronically train the multi-channel deep-learningsegmentation model via backpropagation. Instead, various embodiments ofthe subject innovation are inherently and inextricably tied to computertechnology and cannot be implemented outside of a computing environment(e.g., embodiments of the subject innovation constitute a computerizedtool that can utilize unsupervised modeling to train and/or execute amulti-channel deep-learning segmentation model; such a computerized toolcannot be practicably implemented in any sensible way withoutcomputers).

Moreover, various embodiments of the subject innovation can integrateinto a practical application various teachings described herein relatingto the field of image segmentation. As explained above, existingdeep-learning models that are configured to produce image segmentationsare structured as purely supervised single-channel models. That is, anexisting deep-learning model receives as input only the CT scan that isdesired to be segmented. In contrast, various embodiments of the subjectinnovation can include a computerized tool that can train and/or executea hybrid unsupervised and supervised multi-channel deep-learning modelthat is configured to produce image segmentations. Such a multi-channeldeep-learning model can receive as input both the CT scan that isdesired to be segmented and at least one class probability mask, wherethe computerized tool can electronically derive the at least one classprobability mask from the CT scan via an unsupervised modelingtechnique, such as Gaussian mixture modeling. As explained above, theunsupervised modeling technique can cause the at least one classprobability mask to contain patterns, trends, distributions, and/orrelations that cannot be readily discerned in the CT scan alone. Thus,since the multi-channel model can receive as input the at least oneclass probability mask, the multi-channel model can have access to suchpatterns, trends, distributions, and/or relations, which can enable themulti-channel model to more accurately segment the CT scan and/or to bemore quickly trained, thereby requiring less training data. In starkcontrast, an existing single-channel model does not receive as input theat least one class probability mask, which can cause the single-channelmodel to less accurately segment the CT scan and/or to be less quicklytrained, thereby requiring more training data. Accordingly, variousembodiments of the subject innovation can improve the performance (e.g.,segmentation accuracy, training time) of deep-learning segmentationmodels, which is a concrete and tangible technical improvement, andwhich clearly constitutes a useful and practical application ofcomputers.

Furthermore, various embodiments of the subject innovation can controlreal-world tangible devices based on the disclosed teachings. Forexample, various embodiments of the subject innovation canelectronically update parameters of a real-world multi-channeldeep-learning segmentation model.

It should be appreciated that the herein figures and description providenon-limiting examples of the subject innovation.

FIG. 1 illustrates a block diagram of an example, non-limiting system100 that can facilitate hybrid unsupervised and supervised imagesegmentation in accordance with one or more embodiments describedherein.

As shown, a hybrid unsupervised and supervised image segmentation system102 can have any suitable form of electronic access (e.g., via a wiredand/or wireless electronic connection) to a computed tomography image104 (hereafter “CT image 104”). In various aspects, the CT image 104 candepict any two-dimensional and/or three-dimensional representation ofany suitable anatomical structure and/or any suitable portion of ananatomical structure. Various non-limiting examples of such ananatomical structure can include a blood vessel, an artery, a lumen, abrain, a heart, a kidney, an intestine, an eyeball, a lung, and/or abladder. In various cases, the anatomical structure depicted by the CTimage 104 can belong to a human patient. In various other cases, theanatomical structure depicted by the CT image 104 can belong to anyother suitable organism (e.g., animal, plant).

In various instances, the CT image 104 can have any suitable dataformat. For example, the CT image 104 can, in some cases, be formattedas a two-dimensional matrix and/or array of pixels, where each pixelexhibits a corresponding Hounsfield unit (HU) value. As another example,the CT image 104 can, in other cases, be formatted as athree-dimensional tensor and/or array of voxels, where each voxelexhibits a corresponding HU value. For ease of explanation andillustration, the herein disclosure mainly describes the CT image 104 asa two-dimensional matrix of pixels. However, those having ordinary skillin the art will appreciate that the herein teachings can be readilyapplied to three-dimensional tensors of voxels (e.g., athree-dimensional tensor/array of voxels can be considered as acollective sequence of two-dimensional matrices/arrays of pixels).

In various aspects, the CT image 104 can be electronically generated viaany suitable CT imaging equipment and/or imaging system; any suitablecontrast agent; any suitable clinical protocol; and/or any suitablereconstruction technique. For example, any suitable slicing and/orreconstruction algorithm can be used to create the CT image 104, anysuitable kernel definition can be used to create the CT image 104,and/or any suitable post-processing procedures can be used to augmentthe CT image 104. As a non-limiting example, the CT image 104 can be acontrast-enhanced coronary computed tomography angiogram (CCTA) image ofa vascular structure (e.g., blood vessel) of a patient. Although notexplicitly shown in FIG. 1 , the herein teachings are not limited onlyto the CT image 104, but can instead be applied to any other suitabletype of image generated by any suitable imaging modality (e.g., CT,contrast-enhanced CT, MRI, ultrasound), as desired.

In various instances, it can be desired to obtain an image segmentationof the CT image 104, where the image segmentation can have the samedimensionality (e.g., same number and/or position of pixels) as the CTimage 104. Furthermore, it can be desired that the image segmentationinclude n classes that are related to and/or otherwise pertinent to theanatomical structure depicted in the CT image 104, for any suitablepositive integer n. As explained herein, the hybrid unsupervised andsupervised image segmentation system 102 can facilitate suchfunctionality.

In various embodiments, the hybrid unsupervised and supervised imagesegmentation system 102 can comprise a processor 106 (e.g., computerprocessing unit, microprocessor) and a computer-readable memory 108 thatis operably and/or operatively and/or communicatively connected/coupledto the processor 106. The computer-readable memory 108 can storecomputer-executable instructions which, upon execution by the processor106, can cause the processor 106 and/or other components of the hybridunsupervised and supervised image segmentation system 102 (e.g.,receiver component 110, probability component 112, execution component114) to perform one or more acts. In various embodiments, thecomputer-readable memory 108 can store computer-executable components(e.g., receiver component 110, probability component 112, executioncomponent 114), and the processor 106 can execute thecomputer-executable components.

In various embodiments, the hybrid unsupervised and supervised imagesegmentation system 102 can comprise a receiver component 110. Invarious aspects, the receiver component 110 can electronically receiveand/or otherwise electronically access the CT image 104. In variousinstances, the receiver component 110 can electronically retrieve the CTimage 104 from any suitable database and/or data structure that iselectronically accessible to the receiver component 110 (e.g., a graphdatabase, a relational database, a hybrid database), whether suchdatabase and/or data structure is remote from and/or local to thereceiver component 110. As a non-limiting example, the receivercomponent 110 can electronically receive the CT image 104 from a CTimaging system (and/or any other suitable computing equipment associatedwith the CT imaging system) that is deployed in a clinical and/orlaboratory setting (e.g., deployed in a hospital). In any case, thereceiver component 110 can electronically access the CT image 104, sothat other components of the hybrid unsupervised and supervised imagesegmentation system 102 (e.g., probability component 112, executioncomponent 114) can manipulate and/or otherwise interact with the CTimage 104 (e.g., with a copy of the CT image 104).

In various embodiments, the hybrid unsupervised and supervised imagesegmentation system 102 can comprise a probability component 112. Invarious aspects, the probability component 112 can electronicallygenerate a set of class probability masks by applying an unsupervisedmodeling technique to the CT image 104.

More specifically, the probability component 112 can, in variousinstances, tabulate a frequency distribution of HU values exhibited bythe pixels of the CT image 104. In various cases, the probabilitycomponent 112 can convert this frequency distribution of HU values intoa probability distribution of HU values. In various aspects, theprobability component 112 can then apply the unsupervised modelingtechnique to the probability distribution of HU values. As a specificnon-limiting example, the unsupervised modeling technique can beGaussian mixture modeling.

In various instances, Gaussian mixture modeling can be considered as aniterative procedure by which a given probability distribution isdecomposed into multiple constituent Gaussian distributions (e.g.,multiple bell curves). That is, application of Gaussian mixture modelingto a given probability distribution can yield estimates of theparameters (e.g., means, variances) of two or more bell curves that addup to the given probability distribution. As those having ordinary skillin the art will appreciate, Gaussian mixture modeling can include anysuitable iterative parameter estimation technique to obtain theestimated parameters of the constituent Gaussians, such as expectationmaximization, Markov chain Monte Carlo, moment matching, and/or spectralmethods.

In various aspects, the probability component 112 can implement Gaussianmixture modeling, thereby causing the probability component 112 toestimate HU means and/or HU variances of n constituent Gaussians thatcollectively sum to the probability distribution of the HU valuesexhibited by the CT image 104. That is, the probability component 112can estimate one constituent Gaussian for each of the n desiredsegmentation classes. In other cases, however, the probability component112 can estimate any other suitable number of constituent Gaussians.Moreover, implementation of Gaussian mixture modeling to the probabilitydistribution of HU values exhibited by the CT image 104 can cause theprobability component 112 to assign to each pixel of the CT image 104 nprobabilities respectively corresponding to the n constituent Gaussians.For instance, for each pixel of the CT image 104, a first of the nprobabilities assigned to the pixel can indicate a likelihood that thepixel belongs to the first constituent Gaussian, a second of the nprobabilities assigned to the pixel can indicate a likelihood that thepixel belongs to the second constituent Gaussian, and/or an n-th of then probabilities assigned to the pixel can indicate a likelihood that thepixel belongs to the n-th constituent Gaussian.

In various instances, the probability component 112 can output n classprobability masks based on the assigned probabilities generated by theGaussian mixture modeling. In various cases, a class probability maskcan have the same pixel dimensions and/or positions as the CT image 104,but rather than exhibiting pixel-wise HU values, the class probabilitymask can exhibit pixel-wise probability values indicating likelihood ofbelonging to a corresponding one of the n constituent Gaussians. Forexample, the pixels of the first class probability mask can exhibitprobabilities of belonging to the first constituent Gaussian, ratherthan exhibiting HU values. So, pixels in the first class probabilitymask that have high values (e.g., above any suitable threshold) can beconsidered as having a high chance of belonging to the first constituentGaussian, whereas pixels in the first class probability mask that havelow values (e.g., below any suitable threshold) can be considered ashaving a low chance of belonging to the first constituent Gaussian.Similarly, the pixels of the n-th class probability mask can exhibitprobabilities of belonging to the n-th constituent Gaussian, rather thanexhibiting HU values. So, pixels in the n-th class probability mask thathave high values (e.g., above any suitable threshold) can be consideredas having a high chance of belonging to the n-th constituent Gaussian,whereas pixels in the n-th class probability mask that have low values(e.g., below any suitable threshold) can be considered as having a lowchance of belonging to the n-th constituent Gaussian.

As mentioned above, Gaussian mixture modeling is a mere non-limitingexample of the unsupervised modeling technique. In various cases, theunsupervised modeling technique can be any other suitable unsupervisedprocedure for computing pixel-wise probabilities of belonging todifferent segmentation classes and/or for otherwise clustering pixelsinto different segmentation classes (e.g., fuzzy C-means clustering,K-means modeling, auto-encoder modeling, variational auto-encodermodeling, generative adversarial networks).

In various embodiments, the hybrid unsupervised and supervised imagesegmentation system 102 can comprise an execution component 114. Invarious aspects, the execution component 114 can electronically executea deep-learning segmentation model on the CT image 104 and on at leastone of the n class probability masks produced by the probabilitycomponent 112. In other words, the execution component 114 can feed boththe CT image 104 and at least one of the n class probability masks asinput to the deep-learning segmentation model, and the deep-learningsegmentation model can produce as output an image segmentation of the CTimage 104.

In various cases, the deep-learning segmentation model can exhibit anysuitable artificial neural network architecture. For instance, thedeep-learning model can have any suitable number of neural networklayers, can have any suitable number of neurons in various layers, canimplement any suitable activation functions, and/or can implement anysuitable interneuron-connectivity patterns.

In various cases, the deep-learning segmentation model can be trained inany suitable supervised fashion.

Because the deep-learning segmentation model can be trained in asupervised fashion, and because the deep-learning segmentation model canbe configured to receive as input both the CT image 104 and at least oneof the n class probability masks that are generated by the unsupervisedmodeling technique, the deep-learning segmentation model can beconsidered as a hybrid unsupervised and supervised multi-channel model.This is in contrast to existing single-channel models, which areconfigured to receive as input only the CT image 104 and not at leastone of the n class probability masks. In other words, such existingsingle-channel models do not incorporate a hybrid unsupervised andsupervised architecture. As explained herein, since the deep-learningsegmentation model is configured to receive as input at least one of then class probability masks, the deep-learning segmentation model canexhibit improved segmentation accuracy as compared to existingsingle-channel models. Furthermore, as explained herein, since thedeep-learning segmentation model is configured to receive as input atleast one of the n class probability masks, the deep-learningsegmentation model can be trained with significantly less training dataas compared to existing single-channel models.

FIG. 2 illustrates a block diagram of an example, non-limiting system200 including unsupervised model and class probability masks that canfacilitate hybrid unsupervised and supervised image segmentation inaccordance with one or more embodiments described herein. As shown, thesystem 200 can, in some cases, comprise the same components as thesystem 100, and can further comprise unsupervised modeling 202 and/or aset of class probability masks 204.

In various embodiments, the probability component 112 can electronicallyimplement and/or otherwise electronically apply the unsupervisedmodeling 202 to the CT image 104, thereby yielding the set of classprobability masks 204. The unsupervised modeling 202 can, in variousembodiments, be Gaussian mixture modeling. As mentioned above, Gaussianmixture modeling can be an iterative technique/procedure for estimatingthe means and/or variances of two or more Gaussian distributions, whichtwo or more Gaussian distributions sum to a given probabilitydistribution. In other words, when Gaussian mixture modeling is appliedto a particular probability distribution, the result can be theidentification of any suitable number of bell curves that collectivelyadd together to form the particular probability distribution. Moreover,the result can further include assigning probabilities to each of thedata points in the probability distribution, which probabilitiesindicate likelihoods of belonging to the each of the identified bellcurves. In various cases, Gaussian mixture modeling can implementexpectation maximization in order to estimate parameters (e.g., means,variances) of the constituent bell curves. In various other cases,however, Gaussian mixture modeling can implement any other suitableparameter estimation technique, such as Markov chain Monte Carlo, momentmatching, spectral techniques, and/or any suitable combination thereof.

Accordingly, in various instances, the probability component 112 cantabulate a probability distribution of HU values exhibited by the CTimage 104 and can apply Gaussian mixture modeling to such probabilitydistribution. This can cause the probability component 112 to identify nconstituent Gaussians that collectively sum to such probabilitydistribution. Once the n constituent Gaussians are identified/estimated,the probability component 112 can compute, for each pixel of the CTimage 104, a likelihood of belonging to each of the n constituentGaussians. In various cases, the probability component 112 can thengenerate the set of class probability masks 204 based on thosepixel-wise likelihoods. Since there can be n segmentation classes, theset of class probability masks 204 can likewise include n classprobability masks. This is further explained in non-limiting fashionwith respect to FIGS. 3-6 .

FIG. 3 illustrates an example, non-limiting Hounsfield unit probabilitydistribution exhibited by an example, non-limiting computed tomographyimage in accordance with one or more embodiments described herein.

As explained above, the CT image 104 can be a two-dimensionalmatrix/array of pixels, where each pixel exhibits an HU value.Alternatively, the CT image 104 can be a three-dimensional array ofvoxels, where each voxel exhibits an HU value. In any case, theprobability component 112 can tabulate a probability distribution of HUvalues based on the CT image 104. This can be facilitated as follows.

In various instances, the probability component 112 can begin bycreating a frequency distribution (e.g., bar graph and/or histogram) ofHU values exhibited by the pixels (and/or voxels, as appropriate) of theCT image 104. To create such a frequency distribution of HU values, theprobability component 112 can define an HU bin size of any suitablemagnitude. Moreover, the probability component 112 can identify an HUrange exhibited by the CT image 104. For instance, the HU range can beequal to and/or otherwise based on the difference between the highest HUvalue of the CT image 104 and the lowest HU value of the CT image 104.Based on the HU bin size and based on the HU range, the probabilitycomponent 112 can determine a number of HU bins with which to constructthe frequency distribution of the CT image 104. In various cases, thenumber of HU bins can be equal to and/or otherwise based on the quotientobtained by dividing the HU range by the HU bin size. In variousaspects, each HU bin can represent a corresponding HU interval in the HUrange. In various instances, for each of the HU bins, the probabilitycomponent 112 can count the number of pixels of the CT image 104 thatfall within that HU bin. For example, a pixel can fall within an HU binif that pixel's HU value is within the HU interval represented by thatHU bin. The result of assigning every pixel in the CT image 104 to acorresponding HU bin can be considered as the frequency distribution ofthe HU values exhibited by the CT image 104.

In various aspects, the probability component 112 can then convert thefrequency distribution of HU values into a probability distribution ofHU values. For instance, the probability component 112 can transform thefrequency and/or cardinality of each HU bin into a probability densityby dividing the area of that HU bin (e.g., area can be equal to HUinterval multiplied by frequency) by the total area of all the HU bins.The result can be a probability distribution of HU values exhibited bythe pixels of the CT image 104.

FIG. 3 illustrates a non-limiting example of such a probabilitydistribution of HU values. As shown in FIG. 3 , a graph 300 depicts aprobability distribution 302, where the abscissa (e.g., x-axis) of thegraph 300 represents HU values and where the ordinate (e.g., y-axis)represents probability densities. In various cases, the probabilitydistribution 302 can be considered as illustrating a probability densityof HU values exhibited by the pixels of a non-limiting exampleembodiment of the CT image 104.

For ease of illustration, the probability distribution 302 is shown asbeing continuous. However, this is a non-limiting example. Those havingordinary skill in the art will appreciate that, in various cases, adiscrete probability distribution can be implemented as desired and/orappropriate.

For purposes of explanation, three points (e.g., 304-308) are called outon the probability distribution 302. The point 304 can represent one ormore pixels of the CT image 104 that exhibit a low HU value (e.g., thepoint 304 is on the left-side and thus low-end of the abscissa) and alow probability density (e.g., the point 304 is on the bottom-side andthus low-end of the ordinate). Furthermore, the point 306 can representone or more pixels of the CT image 104 that exhibit an intermediate HUvalue (e.g., the point 306 is in the middle of the abscissa) and a highprobability density (e.g., the point 306 is on the top-side and thushigh-end of the ordinate). Further still, the point 308 can representone or more pixels of the CT image 104 that exhibit a high HU value(e.g., the point 308 is on the right-side and thus high-end of theabscissa) and a low probability density (e.g., the point 308 is on thebottom-side and thus low-end of the ordinate). Given the overall shapeof the probability distribution 302, the probability distribution 302can be interpreted as indicating that the CT image 104, in thisnon-limiting example, includes very many pixels havingintermediate-to-upper-intermediate HU values, relatively few pixels withlow HU values, and even fewer pixels with high HU values.

In various cases, the probability component 112 can electronically applythe unsupervised modeling 202, which can be Gaussian mixture modeling,to the probability distribution 302.

FIG. 4 illustrates how an example, non-limiting Hounsfield unitprobability distribution can be the sum of multiple Gaussiandistributions in accordance with one or more embodiments describedherein. That is, FIG. 4 shows non-limiting and example results that canbe obtained when the probability component 112 electronically appliesGaussian mixture modeling to the probability distribution 302.

For purposes of explanation, suppose that the anatomical structuredepicted by the CT image 104 is a blood vessel, and further suppose thatthere are three segmentation classes: a blood class, a calcificationclass, and/or a background class. That is, n=3 in this non-limitingexample.

As shown in FIG. 4 , since n=3, application of Gaussian mixture modelingto the probability distribution 302 can cause the probability component112 to identify three constituent Gaussians that make up the probabilitydistribution 302: a constituent Gaussian 402, a constituent Gaussian,404, and/or a constituent Gaussian 406. That is, when the constituentGaussian 402, the constituent Gaussian 404, and the constituent Gaussian406 are all added together, the result can be the probabilitydistribution 302. As shown, each of the constituent Gaussians 402-406can be a bell curve (e.g., Gaussian distribution), meaning that each canbe completely defined by a corresponding mean and variance. As mentionedabove, application of Gaussian mixture modeling to the probabilitydistribution 302 can cause the probability component 112 to iterativelyestimate such means/variances. That is, the probability component 112can iteratively estimate a first HU mean and a first HU variance for theconstituent Gaussian 402, a second HU mean and a second HU variance forthe constituent Gaussian 404, and/or a third HU mean and a third HUvariance for the constituent Gaussian 406.

In various instances, since n=3 in this non-limiting example, theprobability component 112 can assign to each pixel of the CT image 104three probability values that respectively correspond to the threeconstituent Gaussians 402-406. More specifically, for each particularpixel, the probability component 112 can identify how far the HU valueof the particular pixel is from the mean (e.g., center) of each of thethree constituent Gaussians 402-406, and the probability component 112can use these distances to compute a first probability of the particularpixel belonging to the constituent Gaussian 402, a second probability ofthe particular pixel belonging to the constituent Gaussian 404, and athird probability of the particular pixel belonging to the constituentGaussian 406. Moreover, such three probabilities can sum to unity (e.g.,each pixel can be assigned to only one of the three constituentGaussians 402-406, and so the sum of the three probabilities for a givenpixel can be 100%). In various cases, the particular pixel can be mostlikely to belong to the constituent Gaussian whose mean is nearest tothe particular pixel. Conversely, the particular pixel can be leastlikely to belong to the constituent Gaussian whose mean is farthest fromthe particular pixel. In this way, distance from the mean of eachconstituent Gaussian can be leveraged to estimate likelihood ofbelonging to each constituent Gaussian.

For example, suppose that a particular pixel has an HU value thatmatches that of the point 304 (e.g., a low HU value). In such case, thatparticular pixel can be assigned three probabilities: a firstprobability indicating the likelihood that the particular pixel belongsto the constituent Gaussian 402, a second probability indicating thelikelihood that the particular pixel belongs to the constituent Gaussian404, and/or a third probability indicating the likelihood that theparticular pixel belongs to the constituent Gaussian 406. In variouscases, these three probabilities can be based on the distance betweenthe HU value at the point 304 and the HU mean of the constituentGaussian 402, based on the distance between the HU value at the point304 and the HU mean of the constituent Gaussian 404, and based on thedistance between the HU value at the point 304 and the HU mean of theconstituent Gaussian 406. Moreover, these three probabilities can sum tounity. As shown, the point 304 is very near to the mean of theconstituent Gaussian 402, is quite far from the mean of the constituentGaussian 404, and is even farther from the mean of the constituentGaussian 406. Accordingly, the first probability can be quite high,while the second probability can be quite small, and the thirdprobability can be even smaller still. In other words, these threeprobabilities can indicate that the particular pixel can be very likelyto belong to the constituent Gaussian 402 and can be very unlikely tobelong to either of the constituent Gaussians 404 or 406.

As another example, suppose that a particular pixel has an HU value thatmatches that of the point 306 (e.g., an intermediate HU value). In suchcase, that particular pixel can be assigned three probabilities: a firstprobability indicating the likelihood that the particular pixel belongsto the constituent Gaussian 402, a second probability indicating thelikelihood that the particular pixel belongs to the constituent Gaussian404, and/or a third probability indicating the likelihood that theparticular pixel belongs to the constituent Gaussian 406. In variouscases, these three probabilities can be based on the distance betweenthe HU value at the point 306 and the HU mean of the constituentGaussian 402, based on the distance between the HU value at the point306 and the HU mean of the constituent Gaussian 404, and based on thedistance between the HU value at the point 306 and the HU mean of theconstituent Gaussian 406. Also, these three probabilities can sum tounity. As shown, the point 306 is quite close to the mean of theconstituent Gaussian 404, is somewhat far from the mean of theconstituent Gaussian 406, and is even farther from the mean of theconstituent Gaussian 402. Accordingly, the second probability can bequite high, while the third probability can be quite small, and thefirst probability can be even smaller. In other words, these threeprobabilities can indicate that the particular pixel can be very likelyto belong to the constituent Gaussian 404 and can be very unlikely tobelong to either of the constituent Gaussians 402 or 406.

As yet another example, suppose that a particular pixel has an HU valuethat matches that of the point 308 (e.g., a high HU value). In suchcase, that particular pixel can be assigned three probabilities: a firstprobability indicating the likelihood that the particular pixel belongsto the constituent Gaussian 402, a second probability indicating thelikelihood that the particular pixel belongs to the constituent Gaussian404, and/or a third probability indicating the likelihood that theparticular pixel belongs to the constituent Gaussian 406. In variouscases, these three probabilities can be based on the distance betweenthe HU value at the point 308 and the HU mean of the constituentGaussian 402, based on the distance between the HU value at the point308 and the HU mean of the constituent Gaussian 404, and based on thedistance between the HU value at the point 308 and the HU mean of theconstituent Gaussian 406. Moreover, these three probabilities can sum tounity. As shown, the point 308 is closest to the mean of the constituentGaussian 406, is far from the mean of the constituent Gaussian 404, andis even farther from the mean of the constituent Gaussian 402.Accordingly, the third probability can be quite high, while the secondprobability can be quite small, and the first probability can be smallerstill. In other words, these three probabilities can indicate that theparticular pixel can be very likely to belong to the constituentGaussian 406 and can be very unlikely to belong to either of theconstituent Gaussians 402 or 404.

In various aspects, the probability component 112 can generate the setof class probability masks 204 based on the above-mentioned computedprobabilities. Since n=3 in this non-limiting example, the set of classprobability masks 204 can include three class probability masks. Thefirst class probability mask can have the same dimensions (e.g., samenumber and/or positions of pixels) as the CT image 104. But rather thanexhibiting pixel-wise (and/or voxel-wise) HU values, the first classprobability mask can exhibit pixel-wise probabilities of belonging tothe constituent Gaussian 402. Accordingly, pixels that exhibit highvalues in the first class probability mask can be considered as havinghigh chances of belonging to the constituent Gaussian 402, while pixelsthat exhibit low values in the first class probability mask can beconsidered as having low chances of belonging to the constituentGaussian 402. Similarly, the second class probability mask can have thesame dimensions (e.g., same number and/or positions of pixels) as the CTimage 104. But rather than exhibiting pixel-wise HU values, the secondclass probability mask can exhibit pixel-wise probabilities of belongingto the constituent Gaussian 404. Thus, pixels that exhibit high valuesin the second class probability mask can be considered as having highchances of belonging to the constituent Gaussian 404, while pixels thatexhibit low values in the second class probability mask can beconsidered as having low chances of belonging to the constituentGaussian 404. Likewise, the third class probability mask can have thesame dimensions (e.g., same number and/or positions of pixels) as the CTimage 104. But rather than exhibiting pixel-wise HU values, the thirdclass probability mask can exhibit pixel-wise probabilities of belongingto the constituent Gaussian 406. So, pixels that exhibit high values inthe third class probability mask can be considered as having highchances of belonging to the constituent Gaussian 406, while pixels thatexhibit low values in the third class probability mask can be consideredas having low chances of belonging to the constituent Gaussian 406.

FIG. 5 illustrates an example, non-limiting computed tomography image500 in accordance with one or more embodiments described herein. Inother words, FIG. 5 illustrates a non-limiting embodiment of the CTimage 104. As shown, in the non-limiting example of FIG. 5 , the CTimage 104 can be a two-dimensional matrix of pixels that depicts aportion of a patient's anatomy including a blood vessel. In variousinstances, as shown, the CT image 104 can include various pixels thatare dark in color, various pixels that are various shades of gray incolor, and/or various pixels that are bright in color. Pixels that aredark in color can be considered as having low HU values, pixels that aregray in color can be considered as having intermediate HU values, and/orpixels that are bright in color can be considered as having high HUvalues. In various aspects, the gray pixels can represent blood flowingthrough the blood vessel, the white pixels can represent calcificationpresent in the blood vessel, and the black pixels can represent abackground of the blood vessel. As those having ordinary skill in theart will appreciate, the color appearance of these features can bemodified by changing characteristics of the function that maps HU valueto gray-scale color value. For viewing convenience only, a boundary 502is superimposed over the CT image 104 to show where blood can flowthrough the blood vessel. Note that the boundary 502 is included only asan explanatory tool for purposes of this disclosure; the boundary 502 isnot part of the CT image 104 and thus is not received/analyzed by anydeep-learning models described herein.

In various cases, it can be desired to segment the CT image 104 shown inFIG. 5 according to three (e.g., n=3) segmentation classes: a bloodclass, a calcification class, and/or a background class. Prior togenerating such an image segmentation, the probability component 112 canapply the unsupervised modeling 202 (e.g., Gaussian mixture modeling) tothe CT image 104 as described above, thereby generating the set of classprobability masks 204. Since n=3 in this non-limiting example, there canbe n=3 class probability masks in the set of class probability masks204, as shown in FIG. 6 .

FIG. 6 illustrates example, non-limiting class probability masks 600generated via Gaussian mixture modeling in accordance with one or moreembodiments described herein. That is, FIG. 6 shows a non-limitingembodiment of the set of class probability masks 204 that can begenerated by the probability component 112 based on the non-limitingexample of the CT image 104 depicted in FIG. 5 .

As shown in FIG. 6 , the set of class probability masks 204 can includethree class probability masks: 602-606. As shown, each of the classprobability masks 602-606 can have the same dimensionality as the CTimage 104 that is depicted in FIG. 5 . In other words, each of the classprobability masks 602-606 can have the same number of pixels arranged inthe same positions as the CT image 104. Again, for viewing convenienceonly, the boundary 502 is superimposed on each of the class probabilitymasks 602-606, so as to help orient the reader.

In various instances, rather than exhibiting pixel-wise HU values, eachof the class probability masks 602-606 can exhibit pixel-wiseprobability values that respectively correspond to n=3 constituentGaussians iteratively estimated by the probability component 112 viaGaussian mixture modeling. More specifically, the class probability mask602 can exhibit pixel-wise probabilities indicating likelihood ofbelonging to a constituent Gaussian A identified by the probabilitycomponent 112, the class probability mask 604 can exhibit pixel-wiseprobabilities indicating likelihood of belonging to a constituentGaussian B identified by the probability component 112, and/or the classprobability mask 606 can exhibit pixel-wise probabilities indicatinglikelihood of belonging to a constituent Gaussian C identified by theprobability component 112.

So, the higher a pixel's value is (e.g., the brighter the pixel is) inthe class probability mask 602, the more likely it is that the pixelbelongs to the constituent Gaussian A. On the other hand, the lower apixel's value is (e.g., the darker the pixel is) in the classprobability mask 602, the less likely it is that the pixel belongs tothe constituent Gaussian A. In other words, the class probability mask602 can be interpreted as indicating that the brightly-colored pixels inthe class probability mask 602 are likely to belong in the samesegmentation class as each other. As recognizable by a medical expert,it can be the case that the constituent Gaussian A represents and/orotherwise corresponds to the blood segmentation class. After all, thebrightly-colored pixels in the class probability mask 602 are the samepixels that are colored in various shades of gray in FIG. 5 , and suchgray pixels in FIG. 5 can represent blood within the depicted bloodvessel. Accordingly, the class probability mask 602 can, in variousinstances, be considered as a blood class probability mask.

Similarly, the higher a pixel's value is (e.g., the brighter the pixelis) in the class probability mask 604, the more likely it is that thepixel belongs to the constituent Gaussian B. On the other hand, thelower a pixel's value is (e.g., the darker the pixel is) in the classprobability mask 604, the less likely it is that the pixel belongs tothe constituent Gaussian B. In other words, the class probability mask604 can be interpreted as indicating that the brightly-colored pixels inthe class probability mask 604 are likely to belong in the samesegmentation class as each other. As recognizable by a medical expert,it can be the case that the constituent Gaussian B represents and/orotherwise corresponds to the calcification segmentation class. Afterall, the brightly-colored pixels in the class probability mask 604 arethe same pixels that are brightly-colored in FIG. 5 , and suchbrightly-colored pixels in FIG. 5 can represent calcified portions ofthe depicted blood vessel. Accordingly, the class probability mask 604can, in various instances, be considered as a calcification classprobability mask.

Likewise, the higher a pixel's value is (e.g., the brighter the pixelis) in the class probability mask 606, the more likely it is that thepixel belongs to the constituent Gaussian C. On the other hand, thelower a pixel's value is (e.g., the darker the pixel is) in the classprobability mask 606, the less likely it is that the pixel belongs tothe constituent Gaussian C. In other words, the class probability mask606 can be interpreted as indicating that the brightly-colored pixels inthe class probability mask 606 are likely to belong in the samesegmentation class as each other. As recognizable by a medical expert,it can be the case that the constituent Gaussian C represents and/orotherwise corresponds to the background segmentation class. After all,the brightly-colored pixels in the class probability mask 606 are thesame pixels that are darkly-colored in FIG. 5 , and such darkly-coloredpixels in FIG. 5 can represent the background outside of the depictedblood vessel. Accordingly, the class probability mask 606 can, invarious instances, be considered as a background class probability mask.

Although the above discussion mainly explains how the probabilitycomponent 112 can generate the class probability masks 204 via theunsupervised modeling 202 when the unsupervised modeling 202 is Gaussianmixture modeling, this is a mere non-limiting example. Those havingordinary skill in the art will appreciate that the unsupervised modeling202 can, in various embodiments, be any other suitable procedure forgenerating the class probability masks 204. As some non-limitingexamples, the unsupervised modeling 202 can be fuzzy C-means clustering,K-means clustering, auto-encoder modeling, variational auto-encodermodeling, generative adversarial networks, and/or any other suitableunsupervised clustering technique.

FIG. 7 illustrates a block diagram of an example, non-limiting system700 including a deep-learning model and a segmentation that canfacilitate hybrid unsupervised and supervised image segmentation inaccordance with one or more embodiments described herein. As shown, thesystem 700 can, in some cases, comprise the same components as thesystem 200, and can further comprise a deep-learning model 702 and/or asegmentation 704.

In various embodiments, the execution component 114 can electronicallyfeed as input both the CT image 104 and at least one of the set of classprobability masks 204 to the deep-learning model 702. In variousaspects, the deep-learning model 702 can then generate the segmentation704, based on both the CT image 104 and the set of class probabilitymasks 204. In various cases, the segmentation 704 can be any suitableimage segmentation of the CT image 104. That is, the segmentation 704can be a pixel-wise mask that indicates to which of the n segmentationclasses each pixel of the CT image 104 belongs.

In various instances, the deep-learning model 702 can exhibit anysuitable artificial neural network architecture. That is, thedeep-learning model 702 can include any suitable number of neuralnetwork layers, can include any suitable number of neurons in variouslayers (e.g., different layers can have different numbers of neurons),can include any suitable activation functions (e.g., softmax, sigmoid,hyperbolic tangent, rectified linear unit), and/or can include anysuitable interneuron connectivity patterns (e.g., forward connections,skip connections, recurrent connections). In some cases, thedeep-learning model 702 can exhibit a U-Net architecture, including anencoder portion (e.g., one or more down-sampling layers) and a decoderportion (e.g., one or more up-sampling layers).

Although the deep-learning model 702 is depicted in FIG. 7 as beinglocal to the execution component 114, this is merely a non-limitingexample. In various instances, the deep-learning model 702 can be remotefrom the execution component 114 and/or the hybrid unsupervised andsupervised image segmentation system 102.

FIG. 8 illustrates a block diagram of an example, non-limiting system800 including a training component that can facilitate hybridunsupervised and supervised image segmentation in accordance with one ormore embodiments described herein. As shown, the system 800 can, in somecases, comprise the same components as the system 700, and can furthercomprise a training component 802 and/or a set of training computedtomography images 804 (hereafter “set of training CT images 804”).

In various aspects, it can be the case that the segmentation 704 is notan accurate image segmentation of the CT image 104, if the deep-learningmodel 702 has not yet been trained. Accordingly, in various instances,the receiver component 110 can electronically access the set of trainingCT images 804 from any suitable data structure, and the trainingcomponent 802 can electronically train the deep-learning model 702 basedon the set of training CT images 804. More specifically, the set oftraining CT images 804 can include any suitable number of training CTimages, where each training CT image in the set of training CT images804 can have the same dimensionality as the CT image 104. Moreover, eachtraining CT image in the set of training CT images 804 can be annotatedwith a ground truth image segmentation. Accordingly, the trainingcomponent 802 can facilitate supervised training of the deep-learningmodel 702, based on the set of training CT images 804.

For example, the parameters (e.g., weights and/or biases) of thedeep-learning model 702 can be randomly initialized. In various cases,for each training CT image in the set of training CT images 804, thefollowing can occur: the probability component 112 can apply theunsupervised modeling 202 (e.g., Gaussian mixture modeling) to thetraining CT image, thereby yielding n training class probability masks;the execution component 114 can feed both the training CT image and atleast one of the n training class probability masks to the deep-learningmodel 702, thereby causing the deep-learning model 702 to output animage segmentation corresponding to the training CT image; and thetraining component 802 can update, via backpropagation, the parametersof the deep-learning model 702 based on a difference (e.g., loss, error)between the image segmentation produced by the deep-learning model 702and the ground truth image segmentation corresponding to the training CTimage. By repeating this procedure for all (e.g., and/or fewer than all,in some cases) of the set of training CT images 804, the parameters ofthe deep-learning model 702 can be optimized to produce sufficientlyaccurate image segmentations. As those having ordinary skill in the artwill appreciate, the training component 802 can, in some cases,implement batch updating of the parameters of the deep-learning model702, as desired.

In various embodiments, because the deep-learning model 702 can beconfigured to receive as input both the CT image 104 and at least one ofthe set of class probability masks 204, and because the set of classprobability masks 204 can be derived via the unsupervised modeling 202,the deep-learning model 702 can be considered as a hybrid unsupervisedand supervised multi-channel segmentation model. In stark contrast, adeep-learning model that is configured to receive as input only the CTimage 104 would be considered as a purely supervised single-channelsegmentation model. The inventors of various embodiments of the subjectinnovation experimentally verified that a hybrid unsupervised andsupervised multi-channel segmentation model as described herein canexhibit greatly improved segmentation accuracy as compared to a purelysupervised single-channel segmentation model. Moreover, the inventors ofvarious embodiments of the subject innovation further experimentallyverified that a hybrid unsupervised and supervised multi-channelsegmentation model as described herein can exhibit greatly reducedtraining time as compared to a purely supervised single-channelsegmentation model.

Specifically, the inventors configured a hybrid unsupervised andsupervised multi-channel segmentation model as described herein toperform image segmentation of CT scans depicting lumina, where theunsupervised modeling 202 was Gaussian mixture modeling. Furthermore,the inventors validated the hybrid unsupervised and supervisedmulti-channel segmentation model using a coronary CT dataset of 36patients from a particular medical site with invasive pressuremeasurement. The lumina were manually segmented for ground-truthcreation. In addition to the hybrid unsupervised and supervisedmulti-channel segmentation model, the inventors also configured asingle-channel segmentation model. The hybrid unsupervised andsupervised multi-channel segmentation model was configured to receive asinput both a CT scan and one or more class probability masks derivedfrom the CT scan via Gaussian mixture modeling. In contrast, thesingle-channel segmentation model was configured to receive as inputonly a CT scan. However, both models utilized an encoder-decoder neuralnetwork architecture (e.g., U-Net). For comparison, a holdout of 6datasets was used for testing for both models. The hybrid unsupervisedand supervised multi-channel segmentation model was trained with 20%less data (24 datasets) compared to 30 datasets used to train thesingle-channel segmentation model. With a statistical significance levelof p<0.05, the inventors found that the Dice similarity coefficient ofthe hybrid unsupervised and supervised multi-channel segmentation modelwas about 0.84, while the Dice similarity coefficient of thesingle-channel segmentation model was about 0.81. Those having ordinaryskill in the art will appreciate that this means that the hybridunsupervised and supervised multi-channel segmentation model exhibited asignificant improvement in segmentation performance as compared to thesingle-channel segmentation model. Furthermore, it must be emphasizedthat the hybrid unsupervised and supervised multi-channel segmentationmodel achieved this marked performance improvement while having beentrained with 20% less data than the single-channel segmentation model.In other words, not only did the hybrid unsupervised and supervisedmulti-channel segmentation model learn to segment CT scans moreaccurately, but the hybrid unsupervised and supervised multi-channelsegmentation model also required less training to achieve such higheraccuracy. Furthermore, the inventors observed that the hybridunsupervised and supervised multi-channel segmentation model was lessdependent on differences and/or variations in reconstructed HU values,as compared to the single-channel segmentation model, which suggeststhat the hybrid unsupervised and supervised multi-channel segmentationmodel generalized more completely than the single-channel segmentationmodel. Accordingly, various embodiments of the subject innovationcertainly constitute concrete and tangible technical improvements in thefield of image segmentation.

FIG. 9 illustrates a flow diagram of an example, non-limitingcomputer-implemented method 900 that can facilitate hybrid unsupervisedand supervised image segmentation in accordance with one or moreembodiments described herein. In various cases, the computer-implementedmethod 900 can be facilitated by the hybrid unsupervised and supervisedimage segmentation system 102.

In various embodiments, act 902 can include receiving, by a device(e.g., 110) operatively coupled to a processor, a computed tomography(CT) scan (e.g., 104).

In various aspects, act 904 can include tabulating, by the device (e.g.,112), a probability distribution (e.g., 302) of Hounsfield unit (HU)values exhibited by the CT scan. In various cases, each pixel of the CTscan can have an associated HU value, and so a frequency distribution,and thus a probability distribution, of such pixel-wise HU values can betabulated.

In various instances, act 906 can include applying, by the device (e.g.,112), Gaussian mixture modeling (e.g., an embodiment of 202) to thetabulated probability distribution, thereby estimating parameters (e.g.,mean and/or variance) of n individual Gaussian distributions (e.g., 402,404, 406) that sum to the tabulated probability distribution, for anysuitable positive integer n. In various cases, the result can be thateach pixel of the CT scan is assigned n probabilities based on thepixel's HU value, with each probability indicating a likelihood that thepixel belongs to a corresponding one of the n individual Gaussiandistributions.

In various aspects, act 908 can include outputting, by the device (e.g.,112), n class probability masks (e.g., 204, and/or 602-606) based onresults of the Gaussian mixture modeling. In various cases, an i-thclass probability mask, for 1≤i≤n, can show the probability of belongingto the i-th Gaussian distribution for all pixels in the CT scan.

In various instances, act 910 can include feeding, by the device (e.g.,114), both the CT scan and at least one of the n class probability masksto a deep-learning segmentation model (e.g., 702) as input. If thedeep-learning segmentation model has been trained, the deep-learningsegmentation model can output an image segmentation (e.g., 704) based onthe CT scan and based on the n class probability masks.

FIG. 10 illustrates a flow diagram of an example, non-limitingcomputer-implemented method 1000 that can facilitate hybrid unsupervisedand supervised image segmentation in accordance with one or moreembodiments described herein. In various cases, the hybrid unsupervisedand supervised image segmentation system 102 can facilitate thecomputer-implemented method 1000.

In various embodiments, act 1002 can include accessing, by a device(e.g., 110) operatively coupled to a processor, a computed tomography(CT) image (e.g., 104) depicting an anatomical structure.

In various aspects, act 1004 can include generating, by the device(e.g., 112) and via an unsupervised modeling technique (e.g., 202), atleast one class probability mask (e.g., 204) of the anatomical structurebased on the CT image.

In various instances, act 1006 can include generating, by the device(e.g., 114) and via a deep-learning model (e.g., 702), an imagesegmentation (e.g., 704) based on the CT image and based on the at leastone class probability mask.

Although not explicitly shown in FIG. 10 , the computer-implementedmethod 1000 can further comprise: accessing, by the device (e.g., 110),a training CT image (e.g., one of 804); generating, by the device (e.g.,112) and via unsupervised modeling technique, at least one trainingclass probability mask based on the training CT image; and training, bythe device (e.g., 802), the deep-learning model based on the training CTimage and based on the at least one training class probability mask.

Although not explicitly shown in FIG. 10 , the unsupervised modelingtechnique can be Gaussian mixture modeling, fuzzy C-means clustering,K-means clustering, a generative adversarial network, and/or anauto-encoder.

Although not explicitly shown in FIG. 10 , the deep-learning model canexhibit a U-Net architecture that includes an encoder portion and adecoder portion.

Although not explicitly shown in FIG. 10 , the anatomical structure canbe a vascular structure.

Although not explicitly shown in FIG. 10 , the vascular structure can bea lumen, and the at least one class probability mask can include a bloodclass probability mask (e.g., 602), a plaque calcification classprobability mask (e.g., 604), or a background class probability mask(e.g., 606).

Although the herein disclosure mainly describes various embodiments ofthe subject innovation where the CT image 104 is a two-dimensionalmatrix of pixels, this is a mere non-limiting example. Those havingordinary skill in the art will appreciate that the CT image 104 can haveany other suitable dimensionality. For instance, in some cases, the CTimage 104 can be a three-dimensional tensor of voxels, rather than atwo-dimensional matrix of pixels. In any case, those having ordinaryskill in the art will understand that the herein teachings can bereadily applied, regardless of the dimensionality of the CT image 104.Those having ordinary skill in the art will further understand that, ifthe CT image 104 is a three-dimensional tensor of voxels, thesegmentation 704 can likewise be a three-dimensional tensor of voxelsthat indicates to which segmentation class each voxel belongs.

Similarly, although the herein disclosure mainly describes variousembodiments of the subject innovation where the unsupervised modeling202 is Gaussian mixture modeling that involves identifying constituenttwo-dimensional Gaussian distributions, this is a mere non-limitingexample. Those having ordinary skill in the art will appreciate thatmultivariate Gaussian mixture modeling can be applied in variousembodiments (e.g., instead of identifying means and variances,multivariate Gaussian mixture modeling can involve identifying centroidsand covariances).

Likewise, although the herein disclosure mainly describes variousembodiments of the subject innovation where the unsupervised modeling202 is applied to an entire probability distribution of HU valuesexhibited by the CT image 104, this is a mere non-limiting example.Those having ordinary skill in the art will appreciate that theunsupervised modeling 202 (e.g., Gaussian mixture modeling) can, invarious cases, be applied to any suitable number of patches of pixelswithin the CT image 104, as desired.

Moreover, although the herein disclosure mainly describes variousembodiments of the subject innovation that generate image segmentationsof the CT image 104, this is a mere non-limiting example. Those havingordinary skill in the art will appreciate that the herein teachings canbe readily applied to any suitable type of medical images (e.g., can beapplied to improve segmentation of computed tomography images, ofmagnetic resonance images, of ultrasound images, of positron emissiontomography images), even if such medical images utilize pixel valuesthat are not Hounsfield units.

Furthermore, although the herein disclosure mainly describes variousembodiments of the subject innovation that apply to the deep-learningmodel 702, this is a mere non-limiting example. Those having ordinaryskill in the art will appreciate that any suitable segmentation modelcan be enhanced by being configured to receive as input at least one ofthe set of class probability masks 204 (e.g., the segmentation model canbe a neural network, a regression model, a support vector machine).

Various embodiments of the subject innovation relate to the automatedsegmentation of CT images (e.g., and/or any other suitable medicalimages). Existing techniques for facilitating automated imagesegmentation involve single-channel models that are trained in asupervised fashion and that are configured to receive as input only theCT image that is desired to be segmented. As described herein, variousembodiments of the subject innovation can provide for significantlyimproved image segmentation performance via the utilization ofunsupervised modeling techniques, such as Gaussian mixture modeling.Specifically, for a CT image that is desired to be segmented,embodiments of the subject innovation can generate, via Gaussian mixturemodeling, one or more class probability masks based on the CT image.Then, various embodiments of the subject innovation can feed both the CTimage and the one or more class probability masks to a multi-channelsegmentation model. As experimentally verified by the inventors, the oneor more class probability masks can allow for better initializationand/or regularization of the multi-channel segmentation model, therebyresulting in a better segmentation performance (e.g., better delineationof vessel boundaries, better estimation of cross-sectional area ofvessels when three-dimensional CT images are involved), as compared toexisting single-channel segmentation models. Moreover, as experimentallyverified by the inventors, the one or more class probability masksfurther allow the multi-channel segmentation model to learn more quicklyand/or with significantly less (e.g., 20% less) training data, ascompared to existing single-channel segmentation models.

To facilitate some of the above-described machine learning aspects ofvarious embodiments of the subject innovation, consider the followingdiscussion of artificial intelligence. Various embodiments of thepresent innovation herein can employ artificial intelligence (AI) tofacilitate automating one or more features of the present innovation.The components can employ various AI-based schemes for carrying outvarious embodiments/examples disclosed herein. In order to provide foror aid in the numerous determinations (e.g., determine, ascertain,infer, calculate, predict, prognose, estimate, derive, forecast, detect,compute) of the present innovation, components of the present innovationcan examine the entirety or a subset of the data to which it is grantedaccess and can provide for reasoning about or determine states of thesystem and/or environment from a set of observations as captured viaevents and/or data. Determinations can be employed to identify aspecific context or action, or can generate a probability distributionover states, for example. The determinations can be probabilistic; thatis, the computation of a probability distribution over states ofinterest based on a consideration of data and events. Determinations canalso refer to techniques employed for composing higher-level events froma set of events and/or data.

Such determinations can result in the construction of new events oractions from a set of observed events and/or stored event data, whetheror not the events are correlated in close temporal proximity, andwhether the events and data come from one or several event and datasources. Components disclosed herein can employ various classification(explicitly trained (e.g., via training data) as well as implicitlytrained (e.g., via observing behavior, preferences, historicalinformation, receiving extrinsic information, and so on)) schemes and/orsystems (e.g., support vector machines, neural networks, expert systems,Bayesian belief networks, fuzzy logic, data fusion engines, and so on)in connection with performing automatic and/or determined action inconnection with the claimed subject matter. Thus, classification schemesand/or systems can be used to automatically learn and perform a numberof functions, actions, and/or determinations.

A classifier can map an input attribute vector, z=(z1, z2, z3, z4, zn),to a confidence that the input belongs to a class, as byf(z)=confidence(class). Such classification can employ a probabilisticand/or statistical-based analysis (e.g., factoring into the analysisutilities and costs) to determinate an action to be automaticallyperformed. A support vector machine (SVM) can be an example of aclassifier that can be employed. The SVM operates by finding ahyper-surface in the space of possible inputs, where the hyper-surfaceattempts to split the triggering criteria from the non-triggeringevents. Intuitively, this makes the classification correct for testingdata that is near, but not identical to training data. Other directedand undirected model classification approaches include, e.g., naïveBayes, Bayesian networks, decision trees, neural networks, fuzzy logicmodels, and/or probabilistic classification models providing differentpatterns of independence, any of which can be employed. Classificationas used herein also is inclusive of statistical regression that isutilized to develop models of priority.

In order to provide additional context for various embodiments describedherein, FIG. 11 and the following discussion are intended to provide abrief, general description of a suitable computing environment 1100 inwhich the various embodiments of the embodiment described herein can beimplemented. While the embodiments have been described above in thegeneral context of computer-executable instructions that can run on oneor more computers, those skilled in the art will recognize that theembodiments can be also implemented in combination with other programmodules and/or as a combination of hardware and software.

Generally, program modules include routines, programs, components, datastructures, etc., that perform particular tasks or implement particularabstract data types. Moreover, those skilled in the art will appreciatethat the inventive methods can be practiced with other computer systemconfigurations, including single-processor or multiprocessor computersystems, minicomputers, mainframe computers, Internet of Things (IoT)devices, distributed computing systems, as well as personal computers,hand-held computing devices, microprocessor-based or programmableconsumer electronics, and the like, each of which can be operativelycoupled to one or more associated devices.

The illustrated embodiments of the embodiments herein can be alsopracticed in distributed computing environments where certain tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules can be located in both local and remote memory storage devices.

Computing devices typically include a variety of media, which caninclude computer-readable storage media, machine-readable storage media,and/or communications media, which two terms are used herein differentlyfrom one another as follows. Computer-readable storage media ormachine-readable storage media can be any available storage media thatcan be accessed by the computer and includes both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer-readable storage media or machine-readablestorage media can be implemented in connection with any method ortechnology for storage of information such as computer-readable ormachine-readable instructions, program modules, structured data orunstructured data.

Computer-readable storage media can include, but are not limited to,random-access memory (RAM), read-only memory (ROM), electricallyerasable programmable read-only memory (EEPROM), flash memory or othermemory technology, compact disk read-only memory (CD ROM), digitalversatile disk (DVD), Blu-ray disc (BD) or other optical disk storage,magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, solid-state drives or other solid-statestorage devices, or other tangible and/or non-transitory media which canbe used to store desired information. In this regard, the terms“tangible” or “non-transitory” herein as applied to storage, memory orcomputer-readable media, are to be understood to exclude onlypropagating transitory signals per se as modifiers and do not relinquishrights to all standard storage, memory or computer-readable media thatare not only propagating transitory signals per se.

Computer-readable storage media can be accessed by one or more local orremote computing devices, e.g., via access requests, queries or otherdata retrieval protocols, for a variety of operations with respect tothe information stored by the medium.

Communications media typically embody computer-readable instructions,data structures, program modules or other structured or unstructureddata in a data signal such as a modulated data signal, e.g., a carrierwave or other transport mechanism, and includes any information deliveryor transport media. The term “modulated data signal” or signals refersto a signal that has one or more of its characteristics set or changedin such a manner as to encode information in one or more signals. By wayof example, and not limitation, communication media include wired media,such as a wired network or direct-wired connection, and wireless mediasuch as acoustic, RF, infrared and other wireless media.

With reference again to FIG. 11 , the example environment 1100 forimplementing various embodiments of the aspects described hereinincludes a computer 1102, the computer 1102 including a processing unit1104, a system memory 1106 and a system bus 1108. The system bus 1108couples system components including, but not limited to, the systemmemory 1106 to the processing unit 1104. The processing unit 1104 can beany of various commercially-available processors. Dual microprocessorsand other multi-processor architectures can also be employed as theprocessing unit 1104.

The system bus 1108 can be any of several types of bus structure thatcan further interconnect to a memory bus (with or without a memorycontroller), a peripheral bus, and a local bus using any of a variety ofcommercially-available bus architectures. The system memory 1106includes ROM 1110 and RAM 1112. A basic input/output system (BIOS) canbe stored in a non-volatile memory such as ROM, erasable programmableread-only memory (EPROM), EEPROM, which BIOS contains the basic routinesthat help to transfer information between elements within the computer1102, such as during startup. The RAM 1112 can also include a high-speedRAM such as static RAM for caching data.

The computer 1102 further includes an internal hard disk drive (HDD)1114 (e.g., EIDE, SATA), one or more external storage devices 1116(e.g., a magnetic floppy disk drive (FDD), a memory stick or flash drivereader, a memory card reader, etc.) and a drive 1120, e.g., such as asolid-state drive, an optical disk drive, which can read or write from adisk 1122, such as a CD-ROM disc, a DVD, a BD, etc. Alternatively, wherea solid-state drive is involved, disk 1122 would not be included, unlessseparate. While the internal HDD 1114 is illustrated as located withinthe computer 1102, the internal HDD 1114 can also be configured forexternal use in a suitable chassis (not shown). Additionally, while notshown in environment 1100, a solid-state drive (SSD) could be used inaddition to, or in place of, an HDD 1114. The HDD 1114, external storagedevice(s) 1116 and drive 1120 can be connected to the system bus 1108 byan HDD interface 1124, an external storage interface 1126 and a driveinterface 1128, respectively. The interface 1124 for external driveimplementations can include at least one or both of Universal Serial Bus(USB) and Institute of Electrical and Electronics Engineers (IEEE) 1394interface technologies. Other external drive connection technologies arewithin contemplation of the embodiments described herein.

The drives and their associated computer-readable storage media providenonvolatile storage of data, data structures, computer-executableinstructions, and so forth. For the computer 1102, the drives andstorage media accommodate the storage of any data in a suitable digitalformat. Although the description of computer-readable storage mediaabove refers to respective types of storage devices, it should beappreciated by those skilled in the art that other types of storagemedia which are readable by a computer, whether presently existing ordeveloped in the future, could also be used in the example operatingenvironment, and further, that any such storage media can containcomputer-executable instructions for performing the methods describedherein.

A number of program modules can be stored in the drives and RAM 1112,including an operating system 1130, one or more application programs1132, other program modules 1134, and program data 1136. All or portionsof the operating system, applications, modules, and/or data can also becached in the RAM 1112. The systems and methods described herein can beimplemented utilizing various commercially-available operating systemsor combinations of operating systems.

Computer 1102 can optionally comprise emulation technologies. Forexample, a hypervisor (not shown) or other intermediary can emulate ahardware environment for operating system 1130, and the emulatedhardware can optionally be different from the hardware illustrated inFIG. 11 . In such an embodiment, operating system 1130 can comprise onevirtual machine (VM) of multiple VMs hosted at computer 1102.Furthermore, operating system 1130 can provide runtime environments,such as the Java runtime environment or the .NET framework, forapplications 1132. Runtime environments are consistent executionenvironments that allow applications 1132 to run on any operating systemthat includes the runtime environment. Similarly, operating system 1130can support containers, and applications 1132 can be in the form ofcontainers, which are lightweight, standalone, executable packages ofsoftware that include, e.g., code, runtime, system tools, systemlibraries and settings for an application.

Further, computer 1102 can be enabled with a security module, such as atrusted processing module (TPM). For instance with a TPM, bootcomponents hash next-in-time boot components, and wait for a match ofresults to secured values, before loading a next boot component. Thisprocess can take place at any layer in the code execution stack ofcomputer 1102, e.g., applied at the application execution level or atthe operating system (OS) kernel level, thereby enabling security at anylevel of code execution.

A user can enter commands and information into the computer 1102 throughone or more wired/wireless input devices, e.g., a keyboard 1138, a touchscreen 1140, and a pointing device, such as a mouse 1142. Other inputdevices (not shown) can include a microphone, an infrared (IR) remotecontrol, a radio frequency (RF) remote control, or other remote control,a joystick, a virtual reality controller and/or virtual reality headset,a game pad, a stylus pen, an image input device, e.g., camera(s), agesture sensor input device, a vision movement sensor input device, anemotion or facial detection device, a biometric input device, e.g.,fingerprint or iris scanner, or the like. These and other input devicesare often connected to the processing unit 1104 through an input deviceinterface 1144 that can be coupled to the system bus 1108, but can beconnected by other interfaces, such as a parallel port, an IEEE 1394serial port, a game port, a USB port, an IR interface, a BLUETOOTH®interface, etc.

A monitor 1146 or other type of display device can be also connected tothe system bus 1108 via an interface, such as a video adapter 1148. Inaddition to the monitor 1146, a computer typically includes otherperipheral output devices (not shown), such as speakers, printers, etc.

The computer 1102 can operate in a networked environment using logicalconnections via wired and/or wireless communications to one or moreremote computers, such as a remote computer(s) 1150. The remotecomputer(s) 1150 can be a workstation, a server computer, a router, apersonal computer, portable computer, microprocessor-based entertainmentappliance, a peer device or other common network node, and typicallyincludes many or all of the elements described relative to the computer1102, although, for purposes of brevity, only a memory/storage device1152 is illustrated. The logical connections depicted includewired/wireless connectivity to a local area network (LAN) 1154 and/orlarger networks, e.g., a wide area network (WAN) 1156. Such LAN and WANnetworking environments are commonplace in offices and companies, andfacilitate enterprise-wide computer networks, such as intranets, all ofwhich can connect to a global communications network, e.g., theInternet.

When used in a LAN networking environment, the computer 1102 can beconnected to the local network 1154 through a wired and/or wirelesscommunication network interface or adapter 1158. The adapter 1158 canfacilitate wired or wireless communication to the LAN 1154, which canalso include a wireless access point (AP) disposed thereon forcommunicating with the adapter 1158 in a wireless mode.

When used in a WAN networking environment, the computer 1102 can includea modem 1160 or can be connected to a communications server on the WAN1156 via other means for establishing communications over the WAN 1156,such as by way of the Internet. The modem 1160, which can be internal orexternal and a wired or wireless device, can be connected to the systembus 1108 via the input device interface 1144. In a networkedenvironment, program modules depicted relative to the computer 1102 orportions thereof, can be stored in the remote memory/storage device1152. It will be appreciated that the network connections shown areexample and other means of establishing a communications link betweenthe computers can be used.

When used in either a LAN or WAN networking environment, the computer1102 can access cloud storage systems or other network-based storagesystems in addition to, or in place of, external storage devices 1116 asdescribed above, such as but not limited to a network virtual machineproviding one or more aspects of storage or processing of information.Generally, a connection between the computer 1102 and a cloud storagesystem can be established over a LAN 1154 or WAN 1156 e.g., by theadapter 1158 or modem 1160, respectively. Upon connecting the computer1102 to an associated cloud storage system, the external storageinterface 1126 can, with the aid of the adapter 1158 and/or modem 1160,manage storage provided by the cloud storage system as it would othertypes of external storage. For instance, the external storage interface1126 can be configured to provide access to cloud storage sources as ifthose sources were physically connected to the computer 1102.

The computer 1102 can be operable to communicate with any wirelessdevices or entities operatively disposed in wireless communication,e.g., a printer, scanner, desktop and/or portable computer, portabledata assistant, communications satellite, any piece of equipment orlocation associated with a wirelessly detectable tag (e.g., a kiosk,news stand, store shelf, etc.), and telephone. This can include WirelessFidelity (Wi-Fi) and BLUETOOTH® wireless technologies. Thus, thecommunication can be a predefined structure as with a conventionalnetwork or simply an ad hoc communication between at least two devices.

FIG. 12 is a schematic block diagram of a sample computing environment1200 with which the disclosed subject matter can interact. The samplecomputing environment 1200 includes one or more client(s) 1210. Theclient(s) 1210 can be hardware and/or software (e.g., threads,processes, computing devices). The sample computing environment 1200also includes one or more server(s) 1230. The server(s) 1230 can also behardware and/or software (e.g., threads, processes, computing devices).The servers 1230 can house threads to perform transformations byemploying one or more embodiments as described herein, for example. Onepossible communication between a client 1210 and a server 1230 can be inthe form of a data packet adapted to be transmitted between two or morecomputer processes. The sample computing environment 1200 includes acommunication framework 1250 that can be employed to facilitatecommunications between the client(s) 1210 and the server(s) 1230. Theclient(s) 1210 are operably connected to one or more client datastore(s) 1220 that can be employed to store information local to theclient(s) 1210. Similarly, the server(s) 1230 are operably connected toone or more server data store(s) 1240 that can be employed to storeinformation local to the servers 1230.

The present invention may be a system, a method, an apparatus and/or acomputer program product at any possible technical detail level ofintegration. The computer program product can include acomputer-readable storage medium (or media) having computer-readableprogram instructions thereon for causing a processor to carry outaspects of the present invention. The computer-readable storage mediumcan be a tangible device that can retain and store instructions for useby an instruction execution device. The computer-readable storage mediumcan be, for example, but is not limited to, an electronic storagedevice, a magnetic storage device, an optical storage device, anelectromagnetic storage device, a semiconductor storage device, or anysuitable combination of the foregoing. A non-exhaustive list of morespecific examples of the computer-readable storage medium can alsoinclude the following: a portable computer diskette, a hard disk, arandom-access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), a staticrandom-access memory (SRAM), a portable compact disc read-only memory(CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk,a mechanically encoded device such as punch-cards or raised structuresin a groove having instructions recorded thereon, and any suitablecombination of the foregoing. A computer-readable storage medium, asused herein, is not to be construed as being transitory signals per se,such as radio waves or other freely propagating electromagnetic waves,electromagnetic waves propagating through a waveguide or othertransmission media (e.g., light pulses passing through a fiber-opticcable), or electrical signals transmitted through a wire.

Computer-readable program instructions described herein can bedownloaded to respective computing/processing devices from acomputer-readable storage medium or to an external computer or externalstorage device via a network, for example, the Internet, a local areanetwork, a wide area network and/or a wireless network. The network cancomprise copper transmission cables, optical transmission fibers,wireless transmission, routers, firewalls, switches, gateway computersand/or edge servers. A network adapter card or network interface in eachcomputing/processing device receives computer-readable programinstructions from the network and forwards the computer-readable programinstructions for storage in a computer-readable storage medium withinthe respective computing/processing device. Computer-readable programinstructions for carrying out operations of the present invention can beassembler instructions, instruction-set-architecture (ISA) instructions,machine instructions, machine dependent instructions, microcode,firmware instructions, state-setting data, configuration data forintegrated circuitry, or either source code or object code written inany combination of one or more programming languages, including anobject oriented programming language such as Smalltalk, C++, or thelike, and procedural programming languages, such as the “C” programminglanguage or similar programming languages. The computer-readable programinstructions can execute entirely on the user's computer, partly on theuser's computer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer can beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection can be made to an external computer (for example, through theInternet using an Internet Service Provider). In some embodiments,electronic circuitry including, for example, programmable logiccircuitry, field-programmable gate arrays (FPGA), or programmable logicarrays (PLA) can execute the computer-readable program instructions byutilizing state information of the computer-readable programinstructions to personalize the electronic circuitry, in order toperform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer-readable program instructions. These computer-readable programinstructions can be provided to a processor of a general purposecomputer, special-purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks. These computer-readable program instructions can also be storedin a computer-readable storage medium that can direct a computer, aprogrammable data processing apparatus, and/or other devices to functionin a particular manner, such that the computer-readable storage mediumhaving instructions stored therein comprises an article of manufactureincluding instructions which implement aspects of the function/actspecified in the flowchart and/or block diagram block or blocks. Thecomputer-readable program instructions can also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational acts to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowcharts and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams can represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks can occur out of theorder noted in the Figures. For example, two blocks shown in successioncan, in fact, be executed substantially concurrently, or the blocks cansometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special-purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special-purpose hardwareand computer instructions.

While the subject matter has been described above in the general contextof computer-executable instructions of a computer program product thatruns on a computer and/or computers, those skilled in the art willrecognize that this disclosure also can be implemented in combinationwith other program modules. Generally, program modules include routines,programs, components, data structures, etc. that perform particulartasks and/or implement particular abstract data types. Moreover, thoseskilled in the art will appreciate that the inventivecomputer-implemented methods can be practiced with other computer systemconfigurations, including single-processor or multiprocessor computersystems, mini-computing devices, mainframe computers, as well ascomputers, hand-held computing devices (e.g., PDA, phone),microprocessor-based or programmable consumer or industrial electronics,and the like. The illustrated aspects can also be practiced indistributed computing environments in which tasks are performed byremote processing devices that are linked through a communicationsnetwork. However, some, if not all aspects of this disclosure can bepracticed on stand-alone computers. In a distributed computingenvironment, program modules can be located in both local and remotememory storage devices.

As used in this application, the terms “component,” “system,”“platform,” “interface,” and the like, can refer to and/or can include acomputer-related entity or an entity related to an operational machinewith one or more specific functionalities. The entities disclosed hereincan be either hardware, a combination of hardware and software,software, or software in execution. For example, a component can be, butis not limited to being, a process running on a processor, a processor,an object, an executable, a thread of execution, a program, and/or acomputer. By way of illustration, both an application running on aserver and the server can be a component. One or more components canreside within a process and/or thread of execution and a component canbe localized on one computer and/or distributed between two or morecomputers. In another example, respective components can execute fromvarious computer-readable media having various data structures storedthereon. The components can communicate via local and/or remoteprocesses such as in accordance with a signal having one or more datapackets (e.g., data from one component interacting with anothercomponent in a local system, distributed system, and/or across a networksuch as the Internet with other systems via the signal). As anotherexample, a component can be an apparatus with specific functionalityprovided by mechanical parts operated by electric or electroniccircuitry, which is operated by a software or firmware applicationexecuted by a processor. In such a case, the processor can be internalor external to the apparatus and can execute at least a part of thesoftware or firmware application. As yet another example, a componentcan be an apparatus that provides specific functionality throughelectronic components without mechanical parts, wherein the electroniccomponents can include a processor or other means to execute software orfirmware that confers at least in part the functionality of theelectronic components. In an aspect, a component can emulate anelectronic component via a virtual machine, e.g., within a cloudcomputing system.

In addition, the term “or” is intended to mean an inclusive “or” ratherthan an exclusive “or.” That is, unless specified otherwise, or clearfrom context, “X employs A or B” is intended to mean any of the naturalinclusive permutations. That is, if X employs A; X employs B; or Xemploys both A and B, then “X employs A or B” is satisfied under any ofthe foregoing instances. Moreover, articles “a” and “an” as used in thesubject specification and annexed drawings should generally be construedto mean “one or more” unless specified otherwise or clear from contextto be directed to a singular form. As used herein, the terms “example”and/or “exemplary” are utilized to mean serving as an example, instance,or illustration. For the avoidance of doubt, the subject matterdisclosed herein is not limited by such examples. In addition, anyaspect or design described herein as an “example” and/or “exemplary” isnot necessarily to be construed as preferred or advantageous over otheraspects or designs, nor is it meant to preclude equivalent exemplarystructures and techniques known to those of ordinary skill in the art.

As it is employed in the subject specification, the term “processor” canrefer to substantially any computing processing unit or devicecomprising, but not limited to, single-core processors;single-processors with software multithread execution capability;multi-core processors; multi-core processors with software multithreadexecution capability; multi-core processors with hardware multithreadtechnology; parallel platforms; and parallel platforms with distributedshared memory. Additionally, a processor can refer to an integratedcircuit, an application specific integrated circuit (ASIC), a digitalsignal processor (DSP), a field programmable gate array (FPGA), aprogrammable logic controller (PLC), a complex programmable logic device(CPLD), a discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. Further, processors can exploit nano-scalearchitectures such as, but not limited to, molecular and quantum-dotbased transistors, switches and gates, in order to optimize space usageor enhance performance of user equipment. A processor can also beimplemented as a combination of computing processing units. In thisdisclosure, terms such as “store,” “storage,” “data store,” datastorage,” “database,” and substantially any other information storagecomponent relevant to operation and functionality of a component areutilized to refer to “memory components,” entities embodied in a“memory,” or components comprising a memory. It is to be appreciatedthat memory and/or memory components described herein can be eithervolatile memory or nonvolatile memory, or can include both volatile andnonvolatile memory. By way of illustration, and not limitation,nonvolatile memory can include read-only memory (ROM), programmable ROM(PROM), electrically programmable ROM (EPROM), electrically erasablePROM (EEPROM), flash memory, or nonvolatile random-access memory (RAM)(e.g., ferroelectric RAM (FeRAM). Volatile memory can include RAM, whichcan act as external cache memory, for example. By way of illustrationand not limitation, RAM is available in many forms such as synchronousRAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double datarate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM(SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM),and Rambus dynamic RAM (RDRAM). Additionally, the disclosed memorycomponents of systems or computer-implemented methods herein areintended to include, without being limited to including, these and anyother suitable types of memory.

What has been described above include mere examples of systems andcomputer-implemented methods. It is, of course, not possible to describeevery conceivable combination of components or computer-implementedmethods for purposes of describing this disclosure, but one of ordinaryskill in the art can recognize that many further combinations andpermutations of this disclosure are possible. Furthermore, to the extentthat the terms “includes,” “has,” “possesses,” and the like are used inthe detailed description, claims, appendices and drawings such terms areintended to be inclusive in a manner similar to the term “comprising” as“comprising” is interpreted when employed as a transitional word in aclaim.

The descriptions of the various embodiments have been presented forpurposes of illustration, but are not intended to be exhaustive orlimited to the embodiments disclosed. Many modifications and variationswill be apparent to those of ordinary skill in the art without departingfrom the scope and spirit of the described embodiments. The terminologyused herein was chosen to best explain the principles of theembodiments, the practical application or technical improvement overtechnologies found in the marketplace, or to enable others of ordinaryskill in the art to understand the embodiments disclosed herein.

What is claimed is:
 1. A system, comprising: a processor that executescomputer-executable components stored in a computer-readable memory, thecomputer-executable components comprising: a receiver component thataccesses a computed tomography (CT) image depicting an anatomicalstructure; a probability component that generates, via an unsupervisedmodeling technique, at least one class probability mask of theanatomical structure based on the CT image; and an execution componentthat generates, via a deep-learning model, an image segmentation basedon the CT image and based on the at least one class probability mask. 2.The system of claim 1, wherein the receiver component accesses atraining CT image, wherein the probability component generates, via theunsupervised modeling technique, at least one training class probabilitymask based on the training CT image, and wherein the computer-executablecomponents further comprise: a training component that trains thedeep-learning model based on the training CT image and based on the atleast one training class probability mask.
 3. The system of claim 1,wherein the unsupervised modeling technique is Gaussian mixturemodeling, fuzzy C-means clustering, K-means clustering, a generativeadversarial network, or an auto-encoder.
 4. The system of claim 1,wherein the deep-learning model exhibits a U-Net architecture thatincludes an encoder portion and a decoder portion.
 5. The system ofclaim 1, wherein the anatomical structure is a vascular structure. 6.The system of claim 5, wherein the vascular structure is a lumen, andwherein the at least one class probability mask includes a blood classprobability mask, a plaque calcification class probability mask, or abackground class probability mask.
 7. The system of claim 1, wherein thedeep-learning model receives as input the CT image and the at least oneclass probability mask.
 8. A computer-implemented method, comprising:accessing, by a device operatively coupled to a processor, a computedtomography (CT) image depicting an anatomical structure; generating, bythe device and via an unsupervised modeling technique, at least oneclass probability mask of the anatomical structure based on the CTimage; and generating, by the device and via a deep-learning model, animage segmentation based on the CT image and based on the at least oneclass probability mask.
 9. The computer-implemented method of claim 8,further comprising: accessing, by the device, a training CT image;generating, by the device and via the unsupervised modeling technique,at least one training class probability mask based on the training CTimage; and training, by the device, the deep-learning model based on thetraining CT image and based on the at least one training classprobability mask.
 10. The computer-implemented method of claim 8,wherein the unsupervised modeling technique is Gaussian mixturemodeling, fuzzy C-means clustering, K-means clustering, a generativeadversarial network, or an auto-encoder.
 11. The computer-implementedmethod of claim 8, wherein the deep-learning model exhibits a U-Netarchitecture that includes an encoder portion and a decoder portion. 12.The computer-implemented method of claim 8, wherein the anatomicalstructure is a vascular structure.
 13. The computer-implemented methodof claim 12, wherein the vascular structure is a lumen, and wherein theat least one class probability mask includes a blood class probabilitymask, a plaque calcification class probability mask, or a backgroundclass probability mask.
 14. The computer-implemented method of claim 8,wherein the deep-learning model receives as input the CT image and theat least one class probability mask.
 15. A computer program product forfacilitating hybrid unsupervised and supervised image segmentation, thecomputer program product comprising a computer-readable memory havingprogram instructions embodied therewith, the program instructionsexecutable by a processor to cause the processor to: access a computedtomography (CT) image depicting an anatomical structure; generate, viaan unsupervised modeling technique, at least one class probability maskof the anatomical structure based on the CT image; and generate, via adeep-learning model, an image segmentation based on the CT image andbased on the at least one class probability mask.
 16. The computerprogram product of claim 15, wherein the program instructions arefurther executable to cause the processor to: access a training CTimage; generate, via the unsupervised modeling technique, at least onetraining class probability mask based on the training CT image; andtrain the deep-learning model based on the training CT image and basedon the at least one training class probability mask.
 17. The computerprogram product of claim 15, wherein the unsupervised modeling techniqueis Gaussian mixture modeling, fuzzy C-means clustering, K-meansclustering, a generative adversarial network, or an auto-encoder. 18.The computer program product of claim 15, wherein the deep-learningmodel exhibits a U-Net architecture that includes an encoder portion anda decoder portion.
 19. The computer program product of claim 15, whereinthe anatomical structure is a vascular structure.
 20. The computerprogram product of claim 19, wherein the vascular structure is a lumen,and wherein the at least one class probability mask includes a bloodclass probability mask, a plaque calcification class probability mask,or a background class probability mask.